Kalman-extended genetic algorithm for search in nonstationary environments with noisy fitness evaluations

In basic genetic algorithm (GA) applications, the fitness of a solution takes a value that is certain and unchanging. There are two classes of problem for which this formulation is insufficient. The first consists of ongoing searches for better solutions in a nonstationary environment, where the expected fitness of a solution changes with time in unpredictable ways. The second class consists of applications in which fitness evaluations are corrupted by noise. For problems belonging to either or both of these classes, the estimated fitness of a solution will have an associated uncertainty. Both the uncertainty due to environmental changes (process noise) and the uncertainty due to noisy evaluations (observation noise) can be reduced, at least temporarily, by re-evaluating existing solutions. The Kalman formulation provides a well-developed formal mechanism for treating uncertainty within the GA framework. It provides the mechanics for determining the estimated fitness and uncertainty when a new solution is generated and evaluated for the first time. It also provides the mechanics for updating the estimated fitness and uncertainty after an existing solution is re-evaluated, and for increasing the uncertainty with the passage of time. A Kalman-extended genetic algorithm (KGA) is developed to determine when to generate a new individual, when to re-evaluate an existing individual, and which one to re-evaluate. This KGA is applied to the problem of maintaining a network configuration with minimized message loss, in which the nodes are mobile, and the transmission over a link is stochastic. As the nodes move, the optimal network changes, but information contained within the population of solutions allows efficient discovery of better-adapted solutions. The ability of the KGA to continually find near-optimal solutions is demonstrated at several levels of process and observation noise. The sensitivity of the KGA performance to several control parameters is explored. Index terms – Genetic algorithm, Kalman filter, adaptive control, network optimization.